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ABSTRACT 

The  research  presented  in  this  paper  considers 
the  impact  of  communication  between  multiple 
operators  controlling  multiple  unmanned  systems. 
First,  communication  between  an  unmanned  aerial 
vehicle  (UAV)  operator  and  an  unmanned  ground 
vehicle  (UGV)  operator  was  coded,  to  account  for 
target  identification  support.  Second,  communication 
was  coded  to  account  for  instances  of  a  UAV 
operator  providing  location  support  to  a  UGV 
operator.  Regression  analyses  revealed  that 

performance  only  improved  with  high  amounts  of 
one  type  of  communication  or  the  other.  When  both 
types  of  communication  occurred  at  the  same  time, 
performance  was  equivalent  to  doing  nothing  at  all. 
Implications  of  these  effects  are  discussed. 

1.  INTRODUCTION 

As  research  into  autonomous  operations  of 
unmanned  systems  develops,  two  different 
relationships  are  beginning  to  emerge  around  the 
human-to-robot  ratio.  First,  research  is  indicating 
that  individual  operators  are  ill-suited  for  handling 
multiple  unmanned  assets  (Chadwick,  2005,  2006). 
Second,  teams  outperform  individual  operators  when 
using  one  or  more  assets  (Burke  &  Murphy,  2004, 
Rehfeld,  Curtis,  Fincannon,  &  Jentsch,  2005).  In 
light  of  these  findings,  it  is  quickly  becoming 
apparent  that  teamwork,  currently  the  norm  in 
unmanned  aerial  vehicle  (UAV)  operations,  is  likely 
to  (a)  remain  a  standard  in  future  UAV  operations 
and  (b)  be  adopted  for  all  other  types  of  unmanned 
systems. 

Research  needs  to  investigate  how  operator 
teams  should  collaborate  with  each  other  when 
controlling  multiple  unmanned  vehicles.  Several 
studies  have  indicated  that  adding  a  second 
unmanned  ground  vehicle  (UGV),  theoretically 
allowing  an  operator  team  to  increase  ground  area 
coverage,  in  reality,  either  has  no  effect  or  actually 
hinders  overall  performance  (Chadwick,  2005; 
Rehfeld  et  ah,  2005).  The  addition  of  a  UAV, 
however,  has  been  found  to  improve  specific 
dimensions  of  performance  (Chadwick,  2005,  2008). 


As  a  result,  this  study  focuses  on  coordination  using 
multiple  heterogeneous  assets  for  reconnaissance  and 
surveillance. 

When  considering  team  effectiveness  and 
performance,  many  studies  have  highlighted  the 
importance  of  attending  to  team  process  (Campion, 
Medsker,  &  Higgs,  1993,  Campion,  Papper,  & 
Medsker,  1996,  Tannenbaum,  Beard,  &  Salas,  1992), 
which  refers  to  the  interaction  between  teammates 
while  collaborating  on  tasks.  Emerging  research  with 
unmanned  systems  has  indicated  that  communication 
is  associated  with  situation  awareness  (Burke  & 
Murphy,  2004),  and  can  interact  with  individual 
differences  to  influence  workload  (Fincannon,  Evans, 
Jentsch,  &  Keebler,  2008b).  Given  this,  the  current 
paper  intends  to  explore  the  importance  of  team 
process  by  focusing  on  its  application  in  the 
operation  of  multiple  heterogeneous  unmanned 
systems. 

1.1  Position  Localization  with  Multiple  Systems 

While  the  use  of  multiple  UGVs  has  not  been 
found  to  improve  team  performance,  the  addition  of 
an  unmanned  aerial  vehicle  (UAV)  to  a  UGV  has 
been  found  to  improve  an  operator’s  performance 
with  respect  to  localizing  targets  in  remote 
environments  (Chadwick,  2005,  2008).  Although 
multiple  explanations  have  been  offered  for  this 
effect,  one  possible  explanation  is  that  the  UAV 
perspective  provides  more  global  “Location  Support’’ 
that  aids  UGV  operators  with  better  understanding 
positions  in  the  environment. 

In  the  context  of  team  effectiveness  while  using 
multiple  robotic  assets,  it  is  important  to  consider 
how  the  UAV  perspective  influences  the  dynamic 
between  UAV  and  UGV  operators.  If  the  UGV 
perspective  is  ill-equipped  to  provide  information 
regarding  location,  it  is  likely  to  expect  that  UAV 
operators  are  uniquely  equipped  to  provide  “Location 
Support”  to  UGV  operators.  While  the  addition  of 
UAV  perspective  has  been  considered  in  the  context 
of  individual  operators,  this  has  not  been  considered 
in  the  context  of  teams.  Given  this,  one  goal  of  the 
current  study  is  to  examine  the  influence  of  UAV 
“Location  Support”  as  a  team  process  behavior. 
Specifically,  we  hypothesized  that  “Location 
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Support”  from  a  UAV  operator  would  improve  the 
UGV  operator’s  ability  to  complete  tasks  and  thus 
improve  the  team’s  performance. 

1.2  Target  Identification  with  Multiple  Systems 

A  second  potential  explanation  for  the  positive 
impact  of  adding  a  UAV  relates  to  the  way  the 
availability  of  the  UAV  can  overcome  inherent  UGV 
operational  limitations.  For  example,  a  UGV  may 
encounter  ground  obstacles  that  can  prevent  its 
operator  from  seeing  certain  objectives.  In  these 
cases,  availability  of  the  aerial  perspective  from  a 
UAV  could  overcome  the  UGV’s  lack  of  a  clear 
view.  Conversely  stated,  a  UAV  might  encounter 
visual  obstructions,  like  tree  cover  or  an  overpass  that 
can  prevent  its  operator  from  being  able  to  observe 
various  objectives.  A  UAV/UGV  operator  team, 
therefore,  may  be  able  to  assist  one  another  in 
identifying  targets  that  would  otherwise  not  be 
observable  by  one  vehicle  or  asset  alone.  We  termed 
this  “Objective  Support”  and  hypothesized  that  it 
would  improve  team  performance. 

1.3  Purpose  &  Hypotheses 

For  the  purposes  of  this  study,  we  were 
concerned  with  the  effects  of  the  aforementioned 
processes  on  team  performance.  Specifically,  we 
investigated  the  relative  contributions  that  (a) 
Location  Support  and  (b)  Objective  Support  had  on 
team  performance  in  a  combined  UAV/UGV 
reconnaissance  task. 

Team  research  dictates  that  team  processes  not 
only  have  a  direct  influence  on  performance,  but  they 
can  also  have  interactive  effects  on  performance 
(Tannenbaum  et  al.,  1992).  As  a  result,  this  study 
also  examines  the  significance  of  interactive  effects 
of  Location  Support  and  Objective  Support 

2.  METHOD 

2.1  Participants 

122  students  from  the  University  of  Central 
Florida  formed  61  two-person  teams.  One  team 
member  was  assigned  to  operate  a  UGV,  and  the 
other  a  UAV.  Team  members  were  located  in 
different  places,  and  collaborated  remotely  to 
perform  reconnaissance  missions  in  a  scaled,  Military 
Operations  in  Urban  Terrain  (MOUT)  environment 
(Ososky,  Evans,  Keebler,  &  Jentsch,  2007). 


2.2  Apparatus 
2.2.1  MOUT  Facility 

The  MOUT  facility  (Figure  1)  is  a  1:35  scale 
replica  of  Al-Najeef  Iraq.  Consisting  of  a  25ft  x  18ft 
area,  this  simulation  allows  us  to  represent 
approximately  four  scaled  city  blocks.  Unlike 
computer  simulations,  our  environment  is  reactive. 
When  participants  drive  the  vehicles  into  objects,  the 
objects  actually  move.  We  believe  this  adds  fidelity 
and  realism  to  the  simulation,  above  what  could  be 
done  in  a  computer  simulation. 


Figure  1.  Scaled  Military  Operations  in  Urban 
Terrain  (MOUT)  environment. 


2.2.2  UGV  &  UAV 

As  discussed  by  Ososky  et  al.  (2007),  the  UGV 
and  UAV  both  consist  of  re-engineered  Remote 
Control  (RC)  vehicles  that  are  coupled  with  micro¬ 
control  boards,  in  order  to  allow  for  remote  control  of 
the  vehicles  via  computer.  The  UGV  is  equipped  with 
two  miniature  wireless  video  cameras.  These  cameras 
are  placed  in  two  locations  on  the  vehicle:  one  as  a 
stationary  front  view,  and  the  other  as  a  moveable 
(approximately  160  degrees)  Reconnaissance, 
Surveillance,  and  Target  Acquisition  (RSTA)  camera. 
The  UAV  consists  of  a  vehicle  that  drives  on  a  track 
hanging  approximately  ten  feet  above  the  facility, 
simulating  a  distance  of  approximately  350  feet  in  the 
air.  The  simulated  UAV  is  equipped  with  one 
camera  that  points  straight  down  from  the  vehicles 
position.  Throughout  the  experiment,  the  vehicles 
could  be  controlled  in  one  of  two  modes:  Full 
Automation  or  Tele-Operation. 

2.2.2. 1  Full  Automation 

To  reliably  simulate  robotic  automation,  trained 
confederates  were  located  in  the  robotic  operations 
room.  Often  referred  to  as  a  “smoke  and  mirrors” 
setup,  this  single  blind  design  allowed  us  to  fully 
script  the  behaviors  of  the  robotic  entities. 


Confederates  were  able  to  see  routes  drawn  on  a  map 
of  our  MOUT  facility  by  the  participants,  and  they 
then  could  drive  the  vehicles  to  the  end  of  each 
segment  (waypoints).  When  confederates  reached  the 
end  of  a  waypoint,  they  stopped  the  vehicles  and 
waited  for  further  commands  from  the  participants. 
From  the  participant’s  point  of  view,  it  seemed  that 
they  were  “programming”  the  robots,  step  by  step,  to 
move  along  a  route  that  they  had  designed  before  the 
mission  started. 

2.2.22  Tele-Operation 

In  tele-operation  mode,  we  allowed  the 
participants  to  take  control  of  the  RC  vehicles,  via 
their  computer  workstations.  In  case  of  hazardous  use 
of  the  vehicles,  confederates  would  over-ride  this 
function.  Participants  were  urged  to  only  use  this 
mode  when  they  need  to  move  the  vehicle  around  a 
hard  to  see  target,  or  go  back  to  a  previous  target.  If 
during  this  operation  the  participants  move  off  track 
from  the  drawn  path,  placing  the  vehicle  back  into 
Full  Automation  will  take  them  to  their  last  waypoint. 

2.2.3  C4I UAV  Operation  Room 

The  C4I  room  (Figure  2)  is  an  office-like 
environment  where  all  UAV  operation  takes  place. 
Consisting  of  a  computer  station  and  multiple 
monitors,  the  UAV  operator,  depending  on  condition, 
can  see  views  from  both  their  own  vehicle  and  from 
the  UGV’s  front  view  camera  and  RSTA  camera.  An 
experimenter  is  present  at  all  times  to  administer 
paperwork,  save  mission  data,  and  answer  questions 
about  how  to  operate  the  system. 


Figure  2.  Display  setup  from  the  C4I  room 


2.2.4  Forward  Observer  Virtual  Foxhole 

The  Forward  Observer  (FO)  Virtual  Foxhole  is  a 
laboratory  room  converted  into  a  simulated,  desert 
embedded  bunker  (Figure  3).  Using  projectors,  the 


back  wall  of  the  room  displays  a  DVTE  (Deployable 
Virtual  Training  Environment)  of  mountainous  desert 
scenery,  to  add  realism  and  immersion  to  the  bunker. 
The  participant  is  seated  in  a  sandbag  enclosure, 
complete  with  a  camouflage  netting  canopy,  and 
battlefield  accoutrements  including  M-16  rifles, 
canteens,  gas  cans  and  other  military  paraphernalia. 
An  experimenter  is  present  at  all  times,  seated  outside 
the  foxhole  in  the  front  of  the  room. 


Figure  3.  Virtual  foxhole  environment 


2.3  Procedure 

2.3.1  Training  &  Practice 

Participants  began  the  experiment  by  filling  out 
informed  consents  and  biographical  data  forms.  After 
they  finished  this  initial  paperwork,  they  were  then 
brought  into  the  C4I  room  for  training,  regardless  of 
the  vehicle  they  were  using.  Participants  were  trained 
through  a  PowerPoint  presentation  that  displayed 
how  to  operate  their  given  vehicle,  how  to  find  and 
identify  targets,  and  how  to  communicate  to  their 
team  mate.  Once  they  had  finished  the  training 
presentation,  they  were  free  to  ask  questions  to  the 
experimenter.  They  were  then  given  10  minutes  to 
study  a  booklet  containing  the  possible  targets  they 
may  encounter  in  the  missions. 

Following  this,  participants  operated  the  vehicles 
through  three  practice  missions.  Each  successive 
mission  added  another  level  of  difficulty,  above  that 
of  the  previous  missions  (i.e.  increase  the  number  of 
objectives,  increase  need  for  coordination  due  to 
targets  being  placed  in  more  difficult  locations). 
From  start  to  finish,  participants  had  approximately 
1.5  hours  of  training  and  practice  before  reaching  the 
final  mission  that  was  used  in  this  analysis. 

2.3.2  Performance  Task 

The  last  mission  was  set  up  in  a  way  that  both 
vehicles  needed  one  another  in  order  to  find  all 
targets.  For  example,  the  UAV  would  be  given  a  map 
with  a  target  that  was  under  a  bridge.  They  would 
need  to  communicate  to  the  UGV  in  order  to  get  a 
view  from  the  ground,  due  to  the  obstruction  from  an 
aerial  view.  Participants  were  given  20  minutes  to 


reach  the  entire  set  of  objectives  and  identify  as  many 
targets  as  they  could. 

2.4  Measures  &  Analysis 

Three  measures  were  used  in  this  analysis.  In 
order  to  assess  performance,  a  percentage  (0%  to 
100%)  score  was  created  for  each  objective.  This 
score  was  then  combined  across  all  six  objectives  to 
create  a  total  percentage  score  (0%  to  600%)  that  was 
used  in  this  analysis. 

Three  raters  read  transcripts  from  team 
interaction  during  the  final  mission,  which  was  used 
to  look  for  specific  team  process  behaviors.  One  of 
these  measures  assessed  target  identification  support. 
As  mentioned  above,  the  final  mission  consisted  of 
objectives  that  could  only  be  reached  by  a  UAV  or  a 
UGV.  This  measure  was  a  count  of  the  number  of 
objectives  where  a  team  said  that  they  would  send  the 
correct  vehicle  to  the  correct  location  to  successfully 
compete  the  mission  (Range  of  0-6).  Analyses 
indicated  that  this  was  a  reliable  assessment 
(ICC=,80;  a=.92). 

The  second  process  behavior  focused  on  location 
support.  Specifically,  raters  were  asked  to  code  the 
number  of  instances  where  a  UAV  operator  explicitly 
told  a  UGV  operator  where  the  UGV  was  located  in 
the  remote  environment.  Analyses  indicated  that  this 
assessment  was  reliable  (ICC=.81;  a=.93). 

3.  RESULTS 

As  illustrated  by  the  correlation  matrix  in  Table 
1,  there  was  a  significant  correlation  between 
Objective  Support  and  Target  Identification 
performance  (r  =  .23).  A  lack  of  significant 
correlations  also  indicated  that  there  was  neither  a 
direct  relationship  between  Objective  Support  and 
Location  Support  nor  target  identification 
performance  and  Location  Support. 

Regression  analyses  revealed  significant  effects 
of  the  team  processes  on  the  number  of  targets 
identified  during  the  course  of  the  mission. 
Specifically,  a  significant  interaction  was  found 
between  location  support  and  objective  support  on 
performance,  F(3,  57)=3.099,  p<.05,  R~=.14.  If  the 
UAV  operator  only  provided  Location  Support,  target 
identification  improved.  If,  instead,  both  teammates 
only  provided  mission  Objective  Support,  target 
identification  improved.  When  both  types  of  support 
were  provided  during  the  course  of  a  mission, 
however,  there  was  no  benefit,  and  the  number  of 
targets  identified  was  equivalent  to  missions  where 
the  teammates  performed  neither  of  the  support 
behaviors. 


Table  1 

Correlation  Matrix  (N=61 ) 


i 

2 

3 

1 .  Objective  Support  [Range: 
0-6] 

1.0 

2.  UAV  Location  Support 
(UAV  Nav) 

.12 

1.0 

3.  Team  Target  Identification 
Score 

.23* 

-.04 

1.0 

Mean 

1.99 

0.42 

1.60 

Standard  Deviation 

1.71 

0.87 

0.95 

*  p<.05,  **p<.01 


Table  2 


Hierarchical  Regression  Analysis  Reporting  the 
Standardized  Coefficient  (ffi  Predicting  the  Total 
Percentage  of  Targets  Correctly  Identified 


Variable 

STEP  1 

STEP  2 

Objective  Support  (ID 

Support) 

.24  + 

37  ** 

UAV  Location  Support  (UAV 
Loc) 

-.07 

.52  + 

Id  Support  X  UAV  Loc 

-.69  * 

R2 

.06 

.14  * 

A  R2 

* 

OO 

O 

-j-p<.10,  *  p<.05,  **p<.01 


Low  High 

Objective  Support 


Low 

Location 

Support 


-  -k  -  High 

Location 

Support 


Figure  4.  Illustration  of  the  Interactive  Effects  of 
Loaction  Support  and  Objective  Support  on  the 
Prediction  of  the  Percentage  of  Targets  Identified 


4.  DISCUSSION 


The  results  illustrate  that  one  of  the  benefits  of 
mixed  UAV/UGV  operator  teams  is  that  one  asset 
can  be  used  to  support  deficiencies  associated  with 
another  asset,  and  vice  versa.  By  providing  improved 
knowledge  of  a  vehicle’s  location  and/or  by  allowing 
one  operator  to  gather  information  on  objectives  that 


his/her  vehicle  normally  could  not  see,  operator 
teams  can  work  together  to  improve  performance. 

The  observed  interaction,  however,  highlighted 
limitations  to  these  benefits  of  team  support.  When 
both  types  of  support  were  given  during  the  course  of 
a  single  mission,  the  benefits  of  providing  these 
supportive  processes  diminished  completely.  As 
performance  was  lowest  for  the  minimal/maximal 
levels  of  communication  and  highest  for  the  moderate 
levels  of  communication,  the  interaction  here  appears 
to  resemble  an  “inverted  U”  relationship  that  is 
commonly  used  to  describe  the  influence  of  workload 
on  performance.  For  the  relationship  in  this  study,  it 
appeared  as  though  there  were  instances  of  too  much 
communication,  which  served  the  purpose  of 
overloading  the  team  as  a  whole,  hindering 
performance.  Alternatively,  teams  might  have 
experienced  overload,  which  was  manifest  in  the 
form  of  the  observed  communication  pattern. 

4.1  Implications 

In  the  context  of  team  performance,  there  are 
several  implications  of  these  findings.  One  of  these 
could  be  to  take  a  technologically  centered  approach 
to  identify  design  principles  that  might  aid  with 
certain  types  of  communication,  in  order  to  improve 
performance.  For  example,  providing  an  aerial 
perspective  for  UGV  operators  working  alone  has 
been  found  to  increase  target  localization 
performance  (Chadwick,  2005,  2008).  In  the  context 
of  team  performance,  providing  a  UGV  operator  with 
video  from  an  UAV  has  been  found  to  reduce  the 
amount  of  localization  support  from  certain  types  of 
UAV  operators  (Fincannon,  Evans,  Jentsch,  & 
Keebler,  2008b).  Given  that  multiple  visual 
perspectives  appear  to  have  this  pattern  of  effects,  it 
would  be  likely  to  expect  visual  manipulation  to  have 
the  effects  of  improving  team  process  and 
performance. 

Another  approach  that  could  be  developed  from 
these  findings  might  apply  to  aspects  of  training.  For 
example,  a  training  program  might  start  by  making 
teams  aware  of  the  pattern  of  effects,  and  could 
possibly  empower  them  by  teaching  strategies  to 
provide  both  types  of  communication  effectively. 
Specific  strategies  could  involve:  learning  how  to 
communicate  effectively,  understanding  when  is  (or 
is  not)  appropriate  to  provide  support,  how  to 
recognize  when  a  teammate  is  overloaded,  and 
knowing  how  much  information  to  provide  at  one 
time.  To  date,  there  is  little  research  exploring  the 
benefits  of  training  programs  on  team  performance 
with  unmanned  systems,  and  future  research  should 
explore  this  option  in  more  depth. 


CONCLUSIONS 

This  study  identifies  how  operators  that  control 
different  types  of  vehicles  can  support  each  other 
during  a  reconnaissance  task.  In  light  of  the 
interaction  highlighting  the  limitations  of  this 
support,  future  research  should  focus  on  finding 
methods  of  providing  both  types  of  support  to 
teammates  without  the  detrimental  effects  observed 
here.  Potential  solutions  could  be  of  a  technical 
nature  (e.g.,  improved  situation  displays)  or  could  be 
procedural  (e.g.,  by  providing  optimized  procedures 
for  the  timing  and  amount  of  support  that  is  given  by 
one  team  member  to  another).  Finally,  team 
members  may  benefit  from  team  coordination 
training,  which  would  allow  them  to  recognize  and 
better  judge  when,  and  what  type  of  support,  is  most 
helpful  to  other  unmanned  vehicle  operators. 
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